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Abstract 

In this article, we will explore why Karlin-McGregor method of using orthogonal 
polynomials in the study of Markov processes was so successful for one dimensional 
nearest neighbor processes, but failed beyond nearest neighbor transitions. We will 
proceed by suggesting and testing possible fixtures. 

1 Introduction 

This paper was influenced by the approaches described in Deift |1] and questions con- 
sidered in Griinbaum [5j. 

The Karlin-McGreogor diagonalization can be used to answer recurrence/transience 
questions, as well as those of probability harmonic functions, occupation times and 
hitting times, and a large number of other quantities obtained by solving various re- 
currence relations, in the study of Markov chains, see [7], [8], [9], [10], [6], [15], [H], 
[3], [12j . However with some exceptions (see |llj ) those were nearest neighbor Markov 
chains on half-line. Griinbaum [5] mentions two main drawbacks to the method as (a) 
"typically one cannot get either the polynomials or the measure explicitly", and (b) 
"the method is restricted to 'nearest neighbour' transition probability chains that give 
rise to tridiagonal matrices and thus to orthogonal polynomials". In this paper we 
attempt to give possible answers to the second question of Griinbaum [5] for general 
reversible Markov chains. In addition, we will consider possible applications of the 
newer methods in orthogonal polynomials such as using Riemann-Hilbert approach, 
see [l], [2] and [13], and their probabilistic interpretations. 

In Section 2, we will give an overview of the Karlin-McGregor method from a naive 
college linear algebra perspective. In 2.3, we will give a Markov chain interpretation to 
the result of Fokas , Its and Kitaev, connecting orthogonal polynomials and Riemann- 
Hilbert problems. Section 3 deals with one dimensional random walks with jumps 
of size < m, the 2m + 1 diagonal operators. There we consider dioganalizing with 
orthogonal functions. In 3.2, as an example we consider a pentadiagonal operator and 
use Plemelj formula, and a two sided interval to obtain the respective diagonalization. 
In Section 4, we use the constructive approach of Deift [1] to produce the Karlin- 
McGregor diagonalization for all irreducible reversible Markov chains. After that, we 
revisit the example from Section 3. 
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2 Eigenvectors of probability operators 

Suppose P is a tridiagonal operator of a onc-dimensional Markov chain on {0, 1, . . . } 
with forward probabilities and backward probabilities q^- Suppose A is an eigenvalue 

/ Qo \ 

Qi 

is the corresponding right eigenvector such that Qq = 1. So 



of P and (A) 



Q2 

V '■■ ) 



Aq = Pq generates the recurrence relation for Qy Then each Qj{X) is a polynomial 
of j-th degree. The Karlin-McGregor method derives the existance of a probability 
distribution tp such that polynomials Qj{\) are orthogonal with respect to ■0. In other 
words, if TT is stationary with ttq = 1 and < •, • >^ is the inner product in L^((i^), then 

< Qii Qj ^ip- 



Thus {y^Qj(A)}j=o,i,... are orthonormal polynomials, where ttj = . Also 

observe from the recurrence relation that the leading coefficient of Qj is ^ 



PO-Pj-l 



Now, A*q^ = P*q-^ implies X^qi = (P*q^)i for each i, and 



Pt{i,j) 



< X Qi,Qj >f=< (P q )i,Qj >f= 

"J 

Therefore 

Pt{i,j) = TTj < X^Qi,Qj >v> 

Since the spectrum of P lies entirely inside [—1, 1] interval, then so is the support 
of tp. Hence, for \z\ > 1, the generating function 



Qi f 

t=o •' 
2.1 Converting to a Jacobi operator 



Qi{X)Qj{X) 
X-z 



dip{X) 



Let bk = 
recurrence relation for q. 



-^^Pk, then bu 



qk+i due to reversibility condition. Thus the 



X\/TT^Qk = QkV^Qk-l + (1 - ^fc - Pk)V^Qk + PkV^Qk+l , 

can be rewritten as 

Xy/T^Qk = h-iV^Qk-i + akV^Qk + hV^Qk+i, 



Therefore 
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is a Jacoby (symmetric triangular with bk > 0) operator. Observe that P is self-adjoint. 

The above approach extends to all reversible Markov chains. Thus every reversible 
Markov operator is equivalent to a self-adjoint operator, and therefore has an all real 
spectrum. 



2.2 Karlin-McGregor: a simple picture 



It is a basic fact from linear algebra that if Ai, . . . , A„ are distinct real eigenvalues of 
an n X n matrix A, and if ui . . . . ,Un and vi, . . . ,Vn are the corresponding left and right 
eigenvectors. Then A diagonalizcs as follows 



UjvJ 



a{A) 



where u{Xj) = Uj, v{Xj) = Vj, spectrum a{A) = {Ai, . . . , A„}, and 



u{X)vT{X) 



n 



u{X)v'^{X) 



UaiA){X) 



Here ?7o-(yl)(A) is the uniform distribution over the spectrum cr{A). 

It is important to observe that the above integral representation is only possible 
if u(A) and v{X) are well defined - each eigenvalue has multiplicity one, i.e. all distinct 
real eigenvalues. As we will see later, this will become crucial for Karlin-McGregor 
diagonalization of reversible Markov chains. The operator for a reversible Markov 
chain is bounded and is equivalent to a self-adjoint operator, and as such has a real 
bounded spectrum. However the eigenvalue multiplicity will determine whether the 
operator's diagonalization can be expressed in a form of a spectral integral. 

Since the spcctrums cr{P) = a{P*), wc will extend the above diagonalization 
identity to the operator P in the separable Hilbert space First, observe that 

u(A) = (ttoQo; TTiQi, • • • ) satisfies 

uP = XP 

due to reversibility. Hence, extending from a finite case to an infinite dimensional space 
/^(M), obtain 



P* = y" AV(A)u(A)dV'(A) = J . 



( T^oQoQo T^iQoQi ■ ■ ■ \ 

TToQlQo TTlQlQl • • • 



where 



The above is the weak limit of 



^(A)= hm Vn(A) 

n— >+oo 



V-nCA) 



n 



u(A)q^(A) 



f4-(A„)(A) , 
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where An is the restriction of P to the first n coordinates, < eg, . . . , e„_i > 

/ 1-po Po ••• \ 

qi 1 - gi - pi pi ' ■ • : 

q2 1 — g2 ■ ■ 



V 



Pn-2 

Qn-l 1 - Qn-l - Pn-1 ) 



Observe that if Qn(A) = then (Qo(A), . . . , Qn-i(A))^ is the corresponding right 
eigenvector of Thus the spectrum of cr(^n) is the roots of 



So 



n 



u(A)q^(A) 



Qn(A) = 



n 



The orthogonahty follows if we plug in t = 0. Since ttqQoQo = 1, "0 should integrate 
to one. 



Example. Simple random walk and Chebyshev polynomials. The Chcbyshcv polyno- 
mials of the first kind are the ones characterizing a one dimensional simple random 
walk on half line, i.e. the ones with generator 



/ 

1 

2 



Pch 



1 











h 






1 

2 



v ■■ 



\ 



J 



So, ro(A) = 1, ri(A) = A and Tfc+i(A) = 2ATfc(A) - rfc_i(A) for A; = 2, 3, . . . . The 
Chebyshev polynomials satisfy the following trigonometric identity: 



Tk{X) = cos(A;cos"^(A)) 



Now, 



V'n(A) = 



n 



E^=o^^r2(A) 



f^{cos(ncos-i(A))=0}(A) , 



where 7r(0) = 1 and 7r(l) = 7r(2) 



2. Here 



t^{cos(ncos-i(A))=0}(A) - ^^{cos-i(A))=^ + ^, fc=0,l,...,n-l} ('^) 
Thus if Xn ~ f/{cos(ncos-l(A))=0}) then Yn = COS-^(X„) ~ t^{JL+S^, jfc=0,l,..,n-l} ^^'^ 

converges weakly to y ~ ^[o,7r] • Hence converges weakly to 



X = cos(y) 



1 
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I.e. ^ 

^^{cos(ncos-i(A))=0}('^) tt-^/I - ^I'^'^l ^''^^^^ 

Also observe that if x = cos (A), then 

rr^rw 1 , oV^ 2/. N 1 , sin((2n - l)x) 
}_^TTkTk{X) = -l + 2^cos {kx)=n-- + — 

k=0 k=0 ^ ' 

Thus ^ 

dil^n{>^) dtpiX) = — ===X[-i,i](A)dA 

TTV 1 — A^ 

2.3 Riemann-Hilbert problem and a generating function 

of Ptihj) 

Let us write ^jT^Qj{X) = kjPj{X), where kj = ^p^ p. q- leading coefficient 

of ^JWjQjiX). and Pj{\) is therefore a m,onic polynomial. 

In preparation for the next step, let w{X) be the probability density function as- 
sociated with the spectral measure dil){\) = w{X)dX on the compact support, 
supp{ip) C [-1, 1] = S. Also let 



denote the Cauchy transform w.r.t. measure if). 
First let us quote the following theorem. 



1 w{x) 
1 



Theorem. [Fokas, Its and Kitaev, 1990] Let 

v{x) = 

be the jump matrix. Then, for any n G {0, 1, 2, . . . }, 

is the unique solution to the Riemann-Hilbert problem with the above jump matrix v{x) 
and S that satisfies the following condition 

"^^"^(^)( V °n)-^ «^ ^-OO. (1) 

The Riemann-Hilbert problem, for an oriented smooth curve S, is the problem of 
finding m{z), analytic in C \ S such that 

m+(z) = m-{z)v{z), for all z eT,, 

where and m_ denote respectively the limit from the left and the limit from the 
right, for the function m, as we approach a point on S. 
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Suppose we are given the weight function w{X) for the Karhn-McGregor orthogonal 
polynomials q. If m^"^(z) is the solution of the Riemann-Hilbert problem as in the 
above theorem, then for \z\ > 1, 



1 



m 



in) 



1 



gi • • • qnQn{z) 

-2- Q„_i(z) 



^27rtz"+ 



PO-Pn-2 



PO-Pn-22' 



■Qn n 
n + 1 "-^0,n 

rGo,n-l(2;) 



3 Beyond nearest neighbor transitions 

Observe that the Chebyshev polynomials were used to diagonalize a simple one di- 
mensional random walk reflecting at the origin. Let us consider a random walk where 
jumps of sizes one and two are equiprobable 



/ 
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! 
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V ••• 
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The above random walk with the reflector at the origin is reversible with 7r(0) = 1 
and 7r(l) = '7r(2) = • • • = 2. The Karlin-McGregor representation with orthogonal 
polynomials will not automatically extend to this case. However this does not rule out 
obtaining a Karlin-McGregor diagonalization with orthogonal functions. 

In the case of the above pentadiagonal Chebyshev operator, some eigenvalues will 
be of geometric multiplicity two as 



P 



Pch + \Pch 



1 



where Pch is the original tridiagonal Chebyshev operator. 



3.1 2m + 1 diagonal operators 

Consider a 2m + 1 diagonal reversible probability operator P. Suppose it is Karlin- 

Qi 

McGregor diagonalizable. Then for a given A G cr(P), let q (A) = 



Q2 

V '■■ J 



once again 



denote the corresponding right eigenvector such that Qq = \. Since the operator is 
more than tridiagonal, we encounter the problem of finding the next m — 1 functions, 
Qi(A) =/xi(A), Q2W =/X2(A),...,Qm-i(A) =/x„j_i(A). 
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Observe that q = qo + qi/^i + • • ■ + qm-iA*m-i) where each qT(A) 



/ Qo,j \ 
Qi,j 

Q2,j 

V '■ J 



solves PqJ = AqJ recurrence relation with the initial conditions 
Qoj(A)=0, g,_i,,(A) = 0, g,j(A) = l, Q,+i,,(A) =0, g^_i,,(A) = 



In other words, q'^(A) = Q{X)fi^ , where Q(A) 



T T T 

Qi ••• Qm-i 



and jd^ 



( 1 \ 
m(A) 



is such that q(A) G /^(M) for each A € cr(P). 



\ Mm-l(A) / 

Let again denote the restriction of P to the first n coordinates, < eo, . . . , e„_i > 
Observe that if (5n(A) = ••• = (5n+m-i(A) = then ((5o(A), . . . , (5n-i(A))^ is the 
corresponding right eigenvector of A^. Thus the spectrum of cr(A„) consists of the 
roots of 



det 



/ Qn,0(A) gn,l(A) 
Qn+l,o(A) (5„+i,i(A) 



Qn,m-l(A) \ 
Qn+l,m-l (A) 



\ Qn+m-l,o(A) Qn+m-l,l(A) Qn+m-l,m-lW / 



3.2 Chebyshev operators 

Let us now return to the example generalizing the simple random walk reflecting at 
the origin. There one step and two step jumps were equally likely. The characteristic 
equation + — 4Xz^ + + z = for the recurrence relation 

Cn+2 + Cn+1 - 4Ac„ + C„_i + C„_2 = 

can be easily solved by observing that if 2: is a solution then so are z and -. The 
solution in radicals is expressed as zi^2 = ri ± i^/T^-Vi and 2:3^4 = 0^2 i i^/T^-r^ , 
where n = ^l±v|±IlA and r2 = ^i^vOT- 

Observe that ri and r2 are the two roots of s{x) = A, where s{x) = + — ^ is 
the polynomial for which 

P = s{Pch) 

In general, the following is true for all operators P that represent symmetric random 
walks reflecting at the origin, and that allow jumps of up to m flights: there is a 
polynomial s{x) such that P = s{Pch) and the roots Zj of the characteristic relation 
in Ac = Pc will lie on a unit circle with their real parts Re{zj) solving s{x) = A. The 
reason for the latter is the symmetry of the corresponding characteristic equation of 
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order 2m, implying j- = Zj, and therefore the characteristic equation for Ac = Pc can 
be rewritten as 

/I r 1 1\ 

A 



1 

z + - 

z 



where 5 [-2 + |] is the Zhukovskiy function. 

In our case, s(x) = + j)'^ — jq, and for A S (— O] , there will be two candidates 
for fj,i{X), 



^+(A) = n 



-1 + V9 + 16A 



and ^_ (A) = r2 



-1 - V9 + 16A 



Taking < arg z < 27r branch of the logarithm log z, and applying Plemelj formula, 
one would obtain 



1 1 
- - + z 2 exp 
4 



' ds 

9_ S — Z 

16 



where ^+(A) = lim^^;,^ 7m(z)>o w(^) and /i_(A) = lim^_;^_ /m(2)<o 

Now, as we defined Hi{z), we can propose the limits of integration to be a contour 
in C consisting of [— ^>0)^ = limgjo {-^ = 2; + «e : [— ^,0)}, and [— ^)0)_ = 
limgj^o {z = X — ie : x G [~^)0)}> and the [0, 1] segment. Then 



A*q' (A)u(A)dV'(A), 



pt 

'[-^.o)-U[-t|,o)^u[o.i] 
where u(A) is defined as before, and 

/ 

#(A) ^ 



2ttJX + 



16 



+ 



X[-9,o)+(^) + X[o,i](A) 



1 



_9_ 

16 



1 



-^+16 



J 



Let us summarize this section as follows. If the structure of the spectrum does 
not allow Karlin-McGregor diagonalization with orthogonal functions over [—1, 1], say 
when there are two values of /^"^(A) for some A, then one may use Plemelj formula to 
obtain an integral diagonalization of P over the corresponding two sided interval. 



4 Spectral Theorem and why orthogonal poly- 
nomials work 

The constructive proofs in the second chapter of Deift [Ij suggest the reason why 
Karlin-McGregor theory of diagonalizing with orthogonal polynomials works for all 
time reversible Markov chains. Using the same logical steps as in [Ij, we can construct 
a map M which assigns a probability measure dip to a reversible transition operator 



P on a countable state space {0, 1, 2, 
one can instead consider 





V ■■ 







}. W.l.o.g. we can assume P is symmetric as 
) ...\ 



P 
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which is symmetric, and its spectrum coinciding with spectrum (t{P) C [—1, 1]. 
Now, for z G C \ M let G{z) = (eo, {P - zIY^cq). Then 

ImG{z) = i [(eo, {P - ziy^e^) - (eo, [P - zl)~^e^)\ = {Im{z))\{P - zI)-^eo\^ 

and therefore G{z) is a Herglotz function, i.e. G{z) is an analytic map from 
{Im{z) > 0} into {Im{z) > 0}, and as all such functions, it can be represented 
as 

/ I s 



G(z)=az + b+ ^ #(s), lm(z)>0 

J-oo \s-z s^ + ly 

In the above representation a > and b are real constants and dip is a Borel measure 
such that 

/ + 00 -j^ 
^ -^d^(.) < OO 

Deift [U uses G{z) = (eo, {P — zI)~^eo) = — ^ + 0(z^^) to show a = in our case, and 

r+co g 

'=L ^'^(^^ 

as well as the uniqueness of dip. Hence 

G{z) = / Imiz) > 

J s z 

The point of all these is to construct the spectral map 
A4 : {reversible Markov operators P} {probability measures ip on [—1, 1] with compact supp{'ilj)} 
The asymptotic evaluation of both sides in 

(eo,(P-z/)-^eo)= / lm{z)>0 
J s- z 

implies 

(eo,P'eo) = J s^d^Pis) 

Until now we were reapplying the logical steps in Deift [T] for the case of reversible 
Markov chains. However, in the original, the second chapter of Deift [1] gives a con- 
structive proof of the following spectral theorem, that summarizes as 

lA : {bounded Jacobi operators on ^ {probability measures ^ on M with compact supp{'ip)^, 
where lA is one-to-one onto. 

Theorem 1. [Spectral Theorem] For every hounded Jacobi operator A there exists 
a unique probability measure ip with compact support such that 



G(z) = (eo,(^-2/)-^eo) = f 

J — c 



dip{x) 
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The spectral map U : A ^ dip is one-to-one onto, and for every f £ L'^{dip), 

{UAU-^f){s) = sf{s) 

in the following sense 



ieo,Af{A)eo) = J 
So suppose P is a reversible Markov chain, then 

M:P^d^ and U'^ : d^ ^ Pa , 
where Pa is a unique Jacobi operator such that 

(eo,P%)= / = (eo,Pieo) 



Now, if Qj{X) are the orthogonal polynomials w.r.t. dip associated with P^, then 
Qj{PA)eo = ej and 

= {ei,ej) = (Qi(PA)eo,Qj(PA)eo) = (Qj(P)eo, Qj(P)eo) 
Thus, if P is irreducible, then fj = Qj{P)eo is an orthonormal basis for Karlin- 



McGregor diagonalization. If we let F 



fo fi 



then 



P* = I (P*ei,ej, 



F 



j\s'Qi{s)Qj{s)di;{s) 



where P^ = P ^. Also Deift [T] provides a way for constructing 

U-^M : P ^ Pa 
Since Pa is a Jacobi operator, it can be represented as 



Now, 



Pa = 

V ■ 

(eo,Peo) = (eo,PAeo) 



/ ao bo 

bo ai 





61 



61 02 



\ 



I 



b,>0 



ao, (eo, P^eo) = (eo, P^eo) = ag + 69 

(eo, P^eo) = (eo, Pleo) = (oq + ^o)ao + (oo + ai)6o 

and (eo, P^eo) = (eo, Pleo) = (ag + b^f + (ao + aifb^ + b^bj 

thus providing us with the coefficients of the Jacobi operator, ao, bo, ai,..., and 
therefore the orthogonal polynomials Qj. 
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Example. Pentadiagonal Chebyshev operator. For the pentadiagonal P that repre- 
sents the symmetric random walk with equiprobable jumps of sizes one and two, 



1 



9 



(eo,Peo) = 0, (eo,P^eo) = -, {eo, P-'eo) = (eo,P^eo) = — , ... 



Thus 



So 



ao = 0, bo 



32' 



64' 



, etc. 



/ 



1 



A 



1 3 

2 8 







and 



go(A) = l, gi(A) = 2A, Q2(A) 



6 



Then applying classical Fourier analysis, one would obtain 

'•2^ de 



1 r 

K(p-./)-%) = -/ ^ 



i[cos(6') +cos(26i)] - z 



1 dV'(s) 



J. s — 

16 



where 



27rA/s + 



16 



+ 



^[-t|,o)(^) 



1 



s + ^ - i 

* ^ 16 4 



1 



^ + ^ + / 



ds 



To obtain the above expression for dip we used the fact that (eo, {P — zl) ^cq) would 
be the same if there were no reflector at zero. 

4.1 Applications of Karlin-McGregor diagonalization 

Let us list some of the possible applications of the diagonalization. 

• One can extract a sharp rate of convergence to a stationary probability distribu- 
tion, if there is one, see Diaconis et. al. [3]. 

• The generator 



G{z) 



F 



• One can use the Fokas, Its and Kitaev results, and benefit from the connection 
between orthogonal polynomials and Riemann-Hilbert problems. 

• One can interpret random walks in random environment as a random spectral 
measure. 
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